Neuro-dynamic programming as a new framework for decision support for deficit irrigation sytems

The great challenge of the agricultural sector is to produce more food and/or more revenue from less water, which can be achieved by optimal irrigation management. A task of primary importance is the problem of intraseasonal irrigation scheduling under limited seasonal water supply. On the intraseasonal level, a limited amount of water is to be distributed over a number of irrigations, taking into account the crops’ response to water stress at different stages during the growing season. Dynamic programming (DP) has been extensively used for optimization of irrigation scheduling problems (Bras and Cordova, 1981; Rao et al., 1988; Sunantara and Ramirez, 1997; Prasad et al., 2006). An alternative approach to calculate optimal irrigation schedules is provided by static optimization techniques such as linear and nonlinear programming (Shang and Mao, 2006; Gorantiwar et al., 2006). Dynamic optimization is a closed-loop optimization strategy designed for obtaining an optimal look up table for selecting – at each stage of the atmosphereplant-soil system during a growing season – the optimal irrigation decision for each possible state of the system. The popularity and success of this technique can be attributed to the fact that nonlinear and stochastic features of scheduling problems can be handled by DP (Bertsekas, 2000). However, it is well known that computational requirements of DP become overwhelming when the number of state and control variables is too large (Bellman and Dreyfus, 1962). For this reason all the studies applying DP for optimal irrigation scheduling have their limitations (Bras and Cordova, 1981; Rao et al., 1988; Sunantara and Ramirez, 1997). A second disadvantage of the classical DP optimization strategies lies in the necessary discretization of the state variables of the water balance models. This limits the predictive reliability of the models significantly which, in turn, affects the computation of the optimal schedules. A neuro-dynamic programming technique (NDP), which overcomes numerous limitations of dynamic programming (DP), is used for determining the optimal irrigation policy in deficit irrigation. This new simulation-based approach combines a broader range of simulation models with optimization algorithm for solving deterministic and stochastic optimization problems. In the context of simulation-based optimization, a simulation model can be thought of as a function (whose explicit form is a black box for the optimizer) that turns input parameters into output performance measures (Gosavi, 2003). The developed neuro-dynamic programming algorithm for single crop intraseasonal scheduling operates together with general water flow and crop growth simulation models. In the contribution, different management schemes are considered and crop-yield functions generated with the NDP optimization algorithm are compared. The paper is organized as follows. In section 2, we review the new Least-Squares Temporal Difference (LSTD) algorithm for calculating the approximate cost-to-go function for the dynamic programming approach. In section 3, a case study involving deficit irrigation of 4 crops is presented to illustrate the new method and we discuss the results, especially crop-yield functions generated with the dynamic simulation-based scheduling algorithm under both, flexible and very restrictive irrigation constraints (see Fig.1 as a first example). In section 4, we offer some conclusions and suggestions for potential stochastic applications for the NDP approach. 0 100 200 300 400 500 600 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Irrigation Depth (mm) R el at iv ev e Yi el d [-] NDP (10 days, 50 mm) NDP (flexible) Figure 1. Normalized seasonal crop production function for maize.